Research on Text Similarity Measurement Hybrid Algorithm with Term Semantic Information and TF-IDF Method

TF-IDF (term frequency-inverse document frequency) is one of the traditional text similarity calculation methods based on statistics. Because does not consider semantic information words, it cannot accurately reflect between texts, and enhanced distinguish documents poorly because extended vectors with similar terms aggravate curse dimensionality. Aiming at this problem, paper advances a hybrid understanding to calculate texts. Based term weighting tree (TSWT) data structure definition from HowNet, firstly discusses preprocess filter process then utilizes those key similarities according weight features whose greater than given threshold. The experimental results show that method better pure aspect accuracy, recall, F1-metric by different K-means clustering methods.